-
-
Notifications
You must be signed in to change notification settings - Fork 15
feat(stackable-versioned)!: Integrate with ConversionReviews #1050
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
@@ -17,6 +17,9 @@ All notable changes to this project will be documented in this file. | |||
- Add `kube_client` crate override to `k8s(crates())` to specify a custom import path. This override | |||
will not be passed to the `#[kube()]` attribute, but will only be available to internal | |||
`#[versioned]` macro code ([#1038]). | |||
- Add `flux-converter`, which adds the `convert` function, which takes a `ConversionReview` and | |||
produces a `ConversionReview` out of it. It creates and uses the needed transitive `.into()` call | |||
chains ([#XXXX]). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
chains ([#XXXX]). | |
chains ([#1050]). |
crates/stackable-versioned-macros/src/codegen/flux_converter.rs
Outdated
Show resolved
Hide resolved
These tests are removed for now, because we are not able to roundtrip CRD conversions without data loss yet. This feature will be re-added in a follow-up PR in an improved form once loss-less roundtripping is supported.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Left comments and suggestions
@@ -1,5 +1,6 @@ | |||
{ | |||
"rust-analyzer.cargo.features": "all", | |||
"rust-analyzer.imports.granularity.group": "crate", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note to self: check if this is the same in our other rust repos:
- operator-templating
- stackablectl
- docker-images (patchable)
- containerdebug
- actions (interu)
- trino-lb
- product-config (any reason this can't just be a crate inside operator-rs?)
- config-utils
There are more, but they are less used.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nothing to action here, this was just me thinking. I will check it out later.
} | ||
} | ||
|
||
impl fmt::Display for Group { | ||
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { | ||
write!(f, "{}", self.0) | ||
f.write_str(self) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Way tidier. None of this nameless {}
business :p
pub use bucket::{S3Bucket, S3BucketVersion}; | ||
pub use connection::{S3Connection, S3ConnectionVersion}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One day we'll update the operators that use these 😁
@@ -176,4 +161,5 @@ impl ToTokens for KubernetesCrateArguments { | |||
#[derive(Clone, Default, Debug, FromMeta)] | |||
pub struct KubernetesConfigOptions { | |||
pub experimental_conversion_tracking: Flag, | |||
pub enable_tracing: Flag, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What does this default to?
I think it should be on by default (it still requires a subscriber/exporter for the generated code to actually be used).
/// This function only returns `Some` if it is a struct. Enums cannot be used to define | ||
/// Kubernetes custom resources. | ||
pub fn generate_kubernetes_item( | ||
pub fn generate_kubernetes_code( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could use a doc-comment here
#[snafu(display("encountered unknown object api version {api_version:?}"))] | ||
UnknownApiVersion { api_version: String }, | ||
|
||
#[snafu(display("failed to deserialize object from json"))] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
#[snafu(display("failed to deserialize object from json"))] | |
#[snafu(display("failed to deserialize object from JSON"))] |
- Add new `try_convert` function to convert objects received via a ConversionReview | ||
- Add new `enable_tracing` option to `#[versioned(k8s(options(...)))]` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- Add new `try_convert` function to convert objects received via a ConversionReview | |
- Add new `enable_tracing` option to `#[versioned(k8s(options(...)))]` | |
- Add new `try_convert` function to convert objects received via a ConversionReview. | |
- Add new `enable_tracing` option to `#[versioned(k8s(options(...)))]`. |
.iter() | ||
.map(|s| s.ident.to_string()) | ||
.collect::<Vec<String>>() | ||
.join("::"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Curious if there is a syn/syn2 method that already does this 🤔
### `#[versioned(k8s(singular = "..."))]` | ||
Set the singular name. Defaults to lowercased .kind value. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Set the singular name. Defaults to lowercased .kind value. | |
Set the singular name. Defaults to lowercased `kind` value. |
# fn main() { | ||
let merged_crd = Foo::merged_crd(FooVersion::V1Beta1).unwrap(); | ||
println!("{}", serde_yaml::to_string(&merged_crd).unwrap()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
println!("{}", serde_yaml::to_string(&merged_crd).unwrap()); | |
println!("{yaml}", yaml = serde_yaml::to_string(&merged_crd).unwrap()); |
This PR adds utilities to convert a CRD from one version into a different version. This is tightly integrated with CRD conversions, which send
ConversionReview
s to registered webhooks and return that review to the K8s apiserver.TLDR
#[versioned(k8s(skip(merged_crd)))]
flag has been removedmerged_crd
is now suffixed withVersion
try_convert
function to convert objects received via a ConversionReviewenable_tracing
option to#[versioned(k8s(options(...)))]
Detailed Overview
There are multiple different parts to make this whole process work. All of these parts are automatically generated by the macro and for the user it is a single function call:
Foo::try_convert
. The individual parts are explained below.Note
The CRD spec currently does not contain any fields. During the development of the conversion mechanism, this is emitted to more easily focus on the newly generated code. It will eventually contain fields to fully test the implementation.
Updated Version Enum
The generated code tries to abstract away as much of the underlying code as possible. For this use-case, a top-level version enum exists. Before this PR, it only provided an easy way to produce a single, merged CRD based on all defined CRD versions via
Foo::merged_crd(Foo::V1)
. The variants described the available versions and forced users to use well-defined versions instead of&str
which can be anything (eg. an invalid version).This PR changes the inner workings of this enum. Instead of only providing plain variants without any data, each variant now holds the appropriate version of the custom resource.
This change enables us to directly deserialize an object of a custom resource into the enum. This then enables us to generate further (in addition to
merged_crd
) top-level abstractions on the enum.With this change it is not possible to use
Self
to specify the stored apiversion viaFoo::merged_crd()
. Instead, a new enum for this purpose only is generated.(De)serialization from/to JSON values
Most (if not all) conversion will be handled automatically by a CRD conversion webhook. Before the K8s apiserver either stores the custom resource in etcd or responds to clients, it will send a
ConversionReview
to the registered webhook. A list of to be converted objects is provided as a JSON object. This is also represented in Rust code, more specifically aserde_json::Value
.As such, the code needs to be able to turn the JSON object into a Rust representation (in the correct version). Once conversion is performed, the opposite operation needs to be performed: turning the Rust struct back into serialized JSON.
This PR introduces two new (private) methods to do this:
Conversion function
Currently, conversion of custom objects (defined by versioned CRDs) is achieved by receiving the ConversionReview via a webhook (which will be run alongside the operator in the future). This ConversionReview is then handed over to the
try_convert
function. Internally, this will construct strictly typed objects based on the current apiVersion. They are then converted to the desired apiVersion and added to the list of converted objects. If all objects were able to be converted, a successful ConversionReview is returned. Otherwise, a failure is indicated....
Partial expanded code